1. Field of the Invention
The present invention relates generally to the processing of digital image sequences and specifically to temporal filtering of digital image sequences.
2. Description of the Background Art
Sequences of digital images often require filtering to remove noise or artifacts that can impair their visual quality. Examples of such sequences arise for instance in applications such as medical imaging, object tracking, pattern recognition, and video compression. Random noise that is introduced during the recording, storage, or transmission of images can degrade portions of the data and thus distort the visual presentation of an image sequence. Furthermore, at least in the case of video compression, other errors or noise in the data may be introduced in order to reduce the number of bits needed to represent the video sequence. Such errors may cause flicker, discontinuities, or other visual artifacts, adversely affecting display of the sequence.
Image filters seek to minimize the visual artifacts caused by such noise and other errors in image sequences by using correlations in surrounding data to attenuate or to remove the data errors. Such filters can operate in either the spatial domain or the temporal domain, or in some cases in both the spatial and temporal domains simultaneously. Spatial filters exploit spatial correlations within a single image to restore noisy data points to close approximations of the underlying source data. Temporal filters use correlations between different images that are temporally close to each other to accomplish the same goal. Spatio-temporal filters use correlations in both the spatial and temporal domains to replace noisy data with smoothed approximations. Some background on spatio-temporal and temporal filters can be found in the survey article “Noise Reduction Filters for Dynamic Image Sequences: A Review,” by James C. Brailean et al and referenced above.
The present application is concerned primarily with temporal, rather than spatial, filtering. As discussed in the Brailean et al reference, a significant advance in temporal filtering concerns the use of motion compensation to properly align matching regions within different images in the presence of motion. For instance, when an object within a video scene moves over a short time span, that object will appear in different locations in consecutive video frames. Since the goal is to use the correlation between the image data in neighboring frames, identifying the pixels in one frame that correspond to a set of pixels in another frame improves the performance of a temporal filter. Yet accurately locating pixels in neighboring frames that correspond to the same object has been a difficult problem.